Pattern recognition

ABSTRACT

In a pattern recognition system, a pattern which is to be recognized later is prescribed in a learning phase. This pattern is detected sequentially, that is to say the informative areas of the pattern are detected and, moreover, the spatial relationship between the areas is also stored. In the recognition phase, a hypothesis which indicates a presumed pattern and, furthermore, indicates where such further prominent areas should be located in the pattern to be recognized if the presumption is correct, is generated on the basis of the acquired data of a first area of a pattern to be recognized, and on the basis of the stored data. Thus, patterns are learned through their location information, on the one hand, and through their spatial relationship to each other, on the other hand, stored and then re-recognized. The application options of the present invention reside, for example, in robotics, text analysis and image analysis, but also in the field of medical technology (for example in automatic tumor detection in mammographies, etc).

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based on and hereby claims priority to PatentApplication No. 19924010.8 filed on May 26, 1999 in Germany, thecontents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates in general to the field of patternrecognition and, more particularly, to a learning method for a patternrecognition system, to a method for re-recognizing at least one pattern,to a pattern recognition system and to the use of such a patternrecognition system.

2. Description of the Related Art

Pattern recognition herein refers to predetermined patterns to bere-recognized which are fed in advance to a technical system or, inother words, that the pattern recognition system is trained. The patternrecognition system is intended to re-recognize later these patterns onwhich it has been trained in advance.

In this case, “pattern” in the meaning of the present invention is to beunderstood as any two-dimensional or multidimensional representation ofsensory impressions. Patterns in the meaning of the present inventioncan therefore naturally be images of physical objects. Further sensoryimpressions can be smell or sound signals. In the case of sound signals,the two-dimensional representation can be, for example, a frequencyspectrum or the amplitude characteristic as a function of time.

There are, of course, many concrete applications of pattern recognition.Mention may be made, as an example, of, on the one hand, robotics, thepattern recognition system in this case serving the purpose of havingthe robot pick up predetermined objects (which in this case representthe trained patterns), for example from an assembly line or the like.

A further possible field of application is represented in general bymedical technology. For example, the pattern recognition system canrecognize tumor diseases on images of medical imaging systems when thepattern recognition system has been trained on typical syndromes oftumors as patterns.

In an application to acoustic signals, a pattern recognition system canrecognize, for example, trained sounds in a noisy spectrum.

A substantial point with regard to the technical implementation of apattern recognition system is the way in which the information that isreproduced in the pattern is fed to the pattern recognition system. Itis known in this case from the prior art to implement such technicalsystems by what is termed a feed-forward approach such as is explained,for example, in Marr, “Vision: A Computational Investigation into thehuman Representation and Processing of visual Information”, New York,Freeman, 1982. Feed-forward means in essence in this case that onlyinformation on the pattern to be recognized is processed, in particularin the recognition phase of pattern recognition. It has emerged in themeantime that this feed-forward approach is inadequate in the case oftechnical implementation in that the resulting processing speeds are tooslow.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide atechnique for pattern recognition which permits a more efficienttechnical implementation.

In accordance with a first aspect, a learning method is provided for apattern recognition system. In this case, at least one pattern to berecognized is provided, that is to say the pattern recognition system isfed information (data) of the pattern, that is to say of thetwo-dimensional representation of sensory impressions. Data which ineach case reproduce areas of the prescribed pattern are then acquired.It is advantageous in this case to select those areas which areparticularly informative. In the case of a two-dimensional image, theseare usually areas which have a high contrast such as, for example,striking discontinuities in the luminance information or colorinformation. Furthermore, the relative spatial relationships between atleast two areas reproduced by the data are detected. In the case of afrequency spectrum, this can be the spacing between two areas, forexample. In the case of a two-dimensional image, this is usually therelative position of the corresponding areas. The detection of thespatial relationships between the areas is performed separately from theacquisition of the actual data of the corresponding areas. The datawhich reproduce the areas of the prescribed pattern, and the spatialrelationships between the areas are then stored.

The acquisition and storage of the data which reproduce the areas of theprescribed pattern, and the detection and storage of the spatialrelationships between the areas can be performed serially in this case.

The pattern can be an image of a physical object. These physical objectscan be, for example, objects which are to be manipulated by a robot.

The at least one prescribed pattern can also be an image, generated byan imaging system, of a syndrome such as, for example a tumor.

In accordance with the invention, a method is also provided forre-recognizing at least one pattern. Data which characterize at leastone pattern to be re-recognized are stored in advance in this case. Thiscan be performed, in particular, using a method as set forth above. Inthe case of the actual re-recognition, data are then acquired from atleast one area of the pattern to be recognized. What is termed ahypothesis is then generated on the basis of the data, stored inadvance, and the acquired data of the at least one area of the patternto be recognized. The hypothesis specifies in this case the patterncharacterized by the stored data which presumably corresponds to thepattern to be recognized. Data of at least one further area of thepattern to be recognized are acquired and compared with the stored datawhich characterize the corresponding area of the presumed pattern. Ifthe data of the at least one further area of the pattern to berecognized substantially match the stored data, which characterize thecorresponding area of the presumed pattern, the pattern to be recognizedis deemed to be identified as the presumed pattern, and thereforere-recognized. “Substantially match” means in this case that in thetechnical implementation a certain identity threshold value (for examplea predetermined percentage of correspondence) is prescribed upon theovershooting of which the corresponding data are assumed to be matching.

The hypothesis can be generated, for example by an artificial neuralnetwork.

The method for re-recognition can be carried out technically withparticular effectiveness when the at least one further area which servesfor verifying the hypothesis is selected on the basis of the hypothesis.The hypothesis can therefore be analyzed as to where a further area ofthe pattern to be recognized has to be present if the present pattern isactually the presumed pattern in this case. For the case in which thedata of the at least one further area of the pattern to be recognized donot substantially match the stored data (and therefore the hypothesisturns out to be false, that is to say the presumed pattern does notcorrespond to that to be recognized), a further hypothesis is generatedand, as already set forth above, is verified with aid of a yet furtherarea.

A pattern recognition system is further provided in accordance with thepresent invention. The system has a memory in which data are storedwhich characterize at least one pattern to be re-recognized. Alsoprovided is a system to acquire the data of at least one area of apattern to be recognized and to generate a hypothesis based on the datain the memory and based on the acquired data of the at least one area ofthe pattern to be recognized, the generated hypothesis specifying apresumed pattern from the patterns which are characterized by the datastored in advance. The acquisition system is designed so that the dataof at least one further area of the pattern to be recognized areacquired and compared with the data in the memory which characterize thecorresponding area of the presumed pattern. The presumed pattern isdeemed to be re-recognized (identified) when the data of the at leastone further area of the pattern to be recognized substantially matchthose data in the memory which characterize the corresponding area ofthe presumed pattern. The hypothesis may be generated by an artificialneural network.

The system can have an analyzer which selects the at least one furtherarea of the pattern to be recognized as a function of the generatedhypothesis. Thus, by contrast with the feed-forward approach of theprior art (see above), this analyzer is an analyzer which operates inaccordance with a top-down technique. In this case, top-down means thatinformation already present, for example information stored in advance,for example in the recognition phase, are also included.

The system designed as set forth above can be used, in particular, fordetecting objects to be manipulated by a robot. It can be used,furthermore, for detecting typical syndromes.

The technical advantage of the technique represented is, in particular,that the detection can be performed serially in the learning and/orrecognition phase. By contrast with a parallel overall detection of theinformation of a pattern to be learned or to be recognized, this serialdetection requires less arithmetic capability and can be effectivelyimplemented with the aid of classical serial computing architecture.

BRIEF DESCRIPTION OF THE DRAWINGS

Further properties, advantages and features of the present inventionwill now be explained in more detail with reference to an exemplaryembodiment and to the figures of the attached drawings, in which:

FIG. 1 is a block diagram of a system according to the invention fordetecting objects on the basis of neurocognitive perceptions, and

FIG. 2 is a flowchart for recognizing objects.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to like elementsthroughout.

Referring to FIG. 1, the aim firstly is to explain the subassemblies ofa system according to the invention for recognizing patterns. Aflowchart which explains the controlled interaction of thesesubassemblies is therefore explained with reference to FIG. 2.

It may be mentioned that, in accordance with the exemplary embodiments,the recognition is performed visually, but that the invention canlikewise be designed on the basis of other sensory perceptions such as,for example, acoustic perceptions.

In this case, the reference numeral denotes in FIG. 1 means forcompletely representing a pattern (object) to be recognized. These meanscan be, for example, a planar image sensor 9. The reference numeral 1denotes in FIG. 1 means which can displace an observation window in themanner of a search light over the entire surface of the planar imagesensor 9. These means can be, for example, means for selectively readingout predetermined sections of the planar image sensor 9. Output signalsof the observation window 1, which therefore reproduce the visualfeatures of the area currently situated in the window or of a section ofthe representation of the object by the planar image sensor 9, arepassed in accordance with the present invention on the one hand to whatis termed a “what” branch 3 and, on the other hand, to what is termed a“where” branch 6. The “what” branch 3 is responsible in this case fordetecting local features such as, for example, edges, structures ofcolors of the corresponding section of the object 9, whereas the “where”branch 6 is responsible for categorically detecting spatialrelationships of the corresponding local features such as set forthabove. For this purpose, the “where” branch is connected to the “what”branch by a line 10 by which the “where” branch 6 is fed thecorresponding detected local features of the “what” branch 3.

The “what” branch 3 and the “where” branch 6 are respectively connectedvia a line 11 or 12, respectively, to a memory 4 which is an associativegraphics memory. The local features and their spatial relationship arestored graphically in the memory 4. This is performed here with regardto a plurality of objects during a training phase of the system. Notuntil after termination of the training phase is the evaluation of thecontent of the memory 4 undertaken in the actual application phase ofthe system.

The displacement of the observation window 1 is driven by an observationwindow movement controller 2 (observation window controller). Theobservation window movement controller 2 executes this movement controlas a function of two fed analysis signals, specifically an analysissignal from the bottom-up analyzer 5, and a second analysis signal froma top-down analyzer 7.

The bottom-up analyzer 5 analyzes the detected local features of the“what” branch 3. By contrast therewith, the top-down analyzer 7 uses ahypothesis generator 8 which is connected, in turn, to the memory 4 anduses the results stored in the memory 4 during the training phase togenerate a hypothesis.

The system illustrated in FIG. 1 is therefore, firstly, capable ofexecuting lower functions, specifically by the bottom-up analyzer 5,which directly evaluates the detected features of the detected “what”branch 3 and cannot use the memory 4. The bottom-up analyzer 5 thereforemakes use only of sensor input signals.

The system illustrated can, for example, execute higher-order functionsby the top-down analyzer 7 and the hypothesis generator 8. The top-downanalyzer 7 and the hypothesis generator 8 are, specifically, connectedto the memory 4 such that they can use feeding stored findings from atraining phase of the system.

The system illustrated solves the problem of limited memory resourcesthat is inherent to the visual recognition of objects by virtue of thefact that a window mechanism 1 is provided for reducing the incomingvisual information, as a result of which the limited informationresources of the system (resources of the corresponding processors) arenot exceeded. Only the information inside the observation window issubjected to further processing at a higher level.

The system illustrated in FIG. 1 has the “what” detection branch 3 andthe “where” detection branch 6, and so object properties and/or spatialproperties can be processed in separate branches. The object propertiesinclude shape, colors and structure of the object to be recognized.

The “where” detection branch 6 for categorical detection of spatialrelationships detects, for example, positions, size and the like of thelocal features detected by the “what” detection branch 3. Whereas, thus,the “what” detection branch 3 is responsible for obtaining primaryproperties of the section of the observation window 1, the “where”detection branch 6 serves for determining categorical spatialrelationships (left, right, etc) between two groups of local featureswhich are assigned to different positions in the observation window.These two types of information (local features of spatial relationshipsbetween them) are graphically stored in an associative memory during alearning phase. The nodes of this diagram respectively store the set oflocal features which has been detected by the “what” detection branch 3at various positions traversed by the observation window 1 in rasterfashion, and the characteristics illustrated in FIG. 1 categoricallystore the spatial relationship between two nodes which have beendetected by the “what” detection branch 3. During the learning phasebefore the actual application phase, only analysis by the bottom-upanalyzer 5 is performed, and so only sensor information is used and mostof the interesting areas of the objects such as, for example, projectingedges, are traversed and analyzed. As a result, an invariantreproduction of an object is stored as a memory record in the memory 4,the memory record being defined by the local features (local edges) andthe corresponding spatial relationships.

The system illustrated in FIG. 1 also permits what is termed a top-downanalysis by the top-down analyzer 7. The top-down analyzer 7 serves inthis case to displace the observation window on the basis of iterativetests and hypotheses which, and this is performed by the hypothesisgenerator 8 on the basis of the findings stored in the memory 4. Inother words, during the actual recognition phase or application phasenot only is use made of a bottom-up analysis by the bottom-up analyzer 5on the basis of sensory information for the displacement of theobservation window 1—rather, the observation window 1 is displaced onthe basis of the information stored in the memory 4 to areas of theobject representation 9 in which the hypothesis generator 8 expectspredetermined local features. In this way, hypotheses which aregenerated by the hypothesis generator 8 are confirmed or rejectediteratively. This analysis/synthesis loop is executed until thehypothesis which the hypothesis generator 8 has generated and thetop-down analyzer 7 has analyzed in relation to the movement of theobservation window 1 is successfully confirmed, something which meansthat the object has been recognized as a whole.

The limited capacity of the memories, processors and the like used forimplementation is not overtaxed by the creation of the varioussubsystems, represented and named above, of the system according to theinvention. The required capacity (resources) is produced, furthermore,according to the invention by the observation window 1, which resemblesa search light and can traverse the object in raster fashion. The “what”subsystem analyzes a primary local features of the section of the objectin the observation window 1, while the spatial relationship between thelocal features can be detected and analyzed by the “where” subsystem.The findings of these two subsystems are stored in the memory 4. Thehypothesis can then be generated on line in the top-down fashion. For anext movement (displacement) of the observation window 1 is thenperformed iteratively on the basis of this hypothesis. When the featureswhich are detected after the displacement of the observation window 1match the features that are to be expected of the hypothesis generatedby the hypothesis generator 8, this means that the object has actuallybeen recognized. By this iterative procedure in conjunction withutilization of the stored findings in the memory 4, the observationwindow 1 is displaced to call up (read out) further information in orderto check whether the object really has been recognized or, in otherwords, whether the features assumed by the hypothesis match the actualfeatures of the object. The system illustrated therefore constitutes anactive visual system for object recognition.

With reference to FIG. 2, the aim now is to explain the interaction ofthe subassemblies of the system of FIG. 1 in more detail. Two phases areessentially executed in this case:

-   -   the learning or training phase, in which all important areas of        a pattern to be recognized are traversed by the observation        window and stored, and    -   the actual recognition phase, in which hypotheses are generated        and verified on the basis of the findings stored in the training        phase. When the hypothesis is confirmed (presumed features in        accordance with the hypothesis being essential identical to the        actual features of the object), the pattern is recognized as        being correct.

The individual steps are now to be described in more detail:

Firstly, the sequence is started in step S1. The area of the patternwhich is currently situated in the section (detection area) of theobservation window is detected in step S2. Spatial relationships of theareas are detected in step S3 on the basis of the findings of the stepS2. The data of the areas and their spatial relationships are stored instep S4. With the aid of the memory content, a check is made in step S5as to whether sufficiently informative and thus all the important areasof the pattern have been detected and stored. For the case in which thecheck of step S5 is negative, the local features are analyzed in thebottom-up fashion in step S6. The observation window is displaced instep S7 as a function of the result of the analysis in step S6, and theprocessing goes back to step S2. Steps S2 to S7 therefore constitute thelearning of training phase to which all important areas of a prescribedpattern to be recognized are traversed by the observation window,detected and stored. Steps S2 to S7 are repeated in this case until theimportant areas of the pattern and their spatial relationships have beendetected and stored.

If the check in step S5 goes positively, the hypothesis is created instep S8 on the basis of the memory content, and specifies the presumeddata in areas which have so far not been traversed/detected. Thehypothesis is analyzed in step S9 (top-down analysis), and theobservation window is displaced in step S10 as a function of the resultof the analysis. A check is made in step S11 as to whether in the newlytraversed and detected area the acquired data match the actual data ofthe pattern in this area in accordance with the hypothesis. If the checkis negative and the current hypothesis is therefore rejected, whichmeans that the pattern has not been recognized, the actual data of theareas and their spatial relationships are acquired and detected in stepS14 and stored, and the processing goes back to step S8.

Alternatively, it is possible, for example given excessively largedeviations between the presumed data of the further area in accordancewith the hypothesis and the actual data of the pattern for the detectionphase to be aborted and the learning or training phase (steps S2 to S7)to be resumed. If in step S11 in the newly traversed and detected areathe presumed data in accordance with the hypothesis match the actualdata of the pattern in this area, this means that the hypothesis hasbeen confirmed and the pattern has therefore been recognized. Thesequence can therefore be terminated in a step S13.

The steps S8 to S14 therefore constitute the actual recognition phase inwhich hypotheses are created and verified on the basis of the findingsstored in the training phase.

Thus, to summarize, a pattern that is to be recognized later, that is tosay a two- or multidimensional representation of sensory impressions, isprescribed in the learning or training phase. This pattern is detectedsequentially, that is to say data of informative features (edges,projections, etc. in the case of a two-dimensional image) areautomatically detected, on the one hand, and the spatial relationshipbetween these areas is also stored, in addition. Areas which havealready been detected (visited) are never detected again anew during thelearning phase. For a given pattern, the learning phase runs until allthe “interesting” areas of the pattern to be detected and learned havebeen traversed.

The aim in the recognition phase is to re-recognize patterns stored inthe learning phase, and this means that the patterns are also to bere-recognized whenever they have been modified within certain limits bycomparison with the originally learned pattern (rotation, deformation,noisiness, . . .). In the recognition phase, a prominent, informativearea of the pattern to be recognized is firstly analyzed. Starting fromthis initial information, a first hypothesis is generated by calibrationwith the stored patterns. This hypothesis thus constitutes a presumedpattern. Consequently, the attention window is displaced on the basis ofthe hypothesis to where further prominent areas are to be present inaccordance with the hypothesis. The hypothesis can change repeatedly inthe course of a recognition phase, since detected areas are furtherprocessed serially and fed to the memory 4. The size of the attentionwindow can, but need not, be varied.

For the case in which a hypothesis turns out to be false in the courseof the learning phase (the corresponding data of the area do not matchthe stored data of the corresponding area of the presumed pattern), thecurrent hypothesis is rejected and the next best hypothesis is verified.Since a hypothesis is created in the recognition phase immediately afterthe detection of the data of a first area of the pattern to berecognized, and, to be more precise, a ranking list of possiblehypotheses is created, the initially generated hypothesis can, ofcourse, be completely incorrect.

The technical advantage of the technique represented is, in particular,that the detection in the learning and/or recognition phase can beperformed serially. By contrast with a parallel overall detection of theinformation of a pattern to be learned or to be recognized, this serialdetection requires less arithmetic capability and can be implementedeffectively with the aid of classical serial computing architectures.

The invention has been described in detail with particular reference topreferred embodiments thereof and examples, but it will be understoodthat variations and modifications can be effected within the spirit andscope of the invention.

1. A pattern recognition system for determining a match between a recognition area of a pattern to be recognized and a reference area of a reference pattern, comprising: storage means for storing prescribed reference areas of a prescribed reference pattern, and prescribed relative spatial reference relationships, each prescribed relative spatial reference relationship describing relative spatial positions between two of the prescribed reference areas; selection means for selecting a first recognition area from a pattern to be recognized; comparing means for comparing the first recognition area with the prescribed reference areas to select a first reference area from the prescribed reference areas of the prescribed reference pattern; first selection means for selecting a second reference area from the prescribed reference areas by using the first reference area and one of the prescribed relative spatial reference relationships which describes a relative position of the first reference area with respect to the second reference area, wherein the second reference area is formulated as a hypothesis for a presumed second recognition area; second selection means for selecting a second recognition area from the pattern to be recognized by using the first recognition area and the one prescribed relative spatial reference relationship which describes the relative position of the first reference area with respect to the second reference area, and determination means for determining a match between the second recognition area and the second selected reference area, wherein said determining of the match determines whether the hypothesis is verified.
 2. The pattern recognition system as claimed in claim 1, wherein the pattern to be recognized is recognized as the prescribed reference pattern if the match exceeds a prescribable threshold, and otherwise the pattern to be recognized is not recognized as the prescribed reference pattern.
 3. A method for determining a match between a reference area from a reference pattern and a recognition area from a pattern to be recognized, the reference pattern having prescribed reference areas and relative spatial reference relationships which in each case describe relative spatial positions between the reference areas, comprising: selecting a first recognition area from the pattern to be recognized; determining a first reference area from the prescribed reference areas by comparing the first recognition area with the prescribed reference areas; determining a second reference area from the prescribed reference areas by using the first reference area and one of the relative spatial reference relationships which describes a relative position of the first reference area with respect to the second reference area, wherein the second reference area is formulated as a hypothesis for a presumed second recognition area; determining a second recognition area from the pattern to be recognized by using the first recognition area and the one relative spatial reference relationship which describes the relative position of the first reference area with respect to the second reference area; and determining a match between the second recognition area and the second selected reference area, wherein said determining of the match determines whether the hypothesis is verified.
 4. The method as claimed in claim 3, wherein the pattern to be recognized is recognized as the reference pattern when a first prescribed measure of matching is achieved, and otherwise the pattern to be recognized is not recognized as the reference pattern.
 5. The method as claimed in claim 4, further comprising, in case of non-recognition of the pattern to be recognized, repeating said determining of the second reference area, the second recognition area and the match using a different one of the relative spatial reference relationships.
 6. The method as claimed in claim 5, wherein said repeating is carried out iteratively until the pattern to be recognized is recognized as the reference pattern, or a second prescribed measure of matching is reached.
 7. The method as claimed in claim 3, wherein the hypothesis is generated by using an artificial neural network.
 8. The method as claimed in claim 3, wherein the pattern to be recognized describes an object to be manipulated by a robot
 9. The method as claimed in claim 3, wherein the pattern to be recognized is an image, generated by an imaging system, of a clinical picture. 